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Abstract. We consider Reinforced Random Walks where transitions proba- 
bilities are a function of the proportions of times the walk has traversed an 
edge. We give conditions for recurrence or transience. A phase transition is 
observed, similar to Pemantle |Peml| on trees. 
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1. Introduction 

Let G = (V, E) be a graph with V the set of vertices and E the set of edges. 
The graph distance is denoted by d. Linearly reinforced random walks (X n , n > 0) 
are nearest neighbor walks on G (i.e. X n S V and (X n , X n+ i) E E) whose laws are 
defined as follows: denote by (^ r n) n >o the natural filtration associated to (X n ,n > 
0) and for (x, y) S E, set 



In (X"> 



with ao(x, y) > and A > 0. Then for (x. y) E E and on the event {X n = x}, 
V(X n+1 = y | T n ) = an{x ' V) - 

l^{z\(x,z)eE} a n\X, Z) 

When E is non-oriented (i.e. (x, y) E E implies (y,x) E E), for (x,y) E E, set 

n 

b n (x,y) = b (x,y) + A l{(x i - 1 ,x i )e{(x, y Uy,x)}}: 

i=l 

with bo(x, y) = bo(y, x) > and A > 0. The law of an undirected reinforced random 
walk is such that for (x, y) E E, on the event {X n = x}, 

nx n+1 = y \r n )= K ^ y) h 

2^{z\(x,z)£E} °n{x, Z) 

The law of a directed linearly reinforced random walk is the same as the law 
of a random walk in a random environment. Indeed it is equivalent to attach 
independent Polya urns to all sites and then De Finetti Theorem implies that 
it is equivalent to attach independent random probability vectors to each site. 
These probability vectors give the transition probabilities for the walk (when located 
at this site). When the graph is a non-oriented tree, the undirected reinforced 
random walk, with initial position p, has the same law as a directed reinforced 
random walk on G with ao(x, y) — bo(x,y) if d{p,y) = d{p,x) + 1 and ao(x, y) — 

l 
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bo{x,y) + A if d(p,y) = d(p,x) — 1 and reinforcement parameter 2A, or with 
a (x, y) = b (x, y)/(2A) if d(p, y) = d(p, x) + 1 and a (x, y) = {b (x, y) + A)/(2A) 
if d(p, y) = d(p, x) — 1 and reinforcement parameter 1. This representation was first 
observed by Coppersmith and Diaconis [CDiaJ. 

For this class of models results on random walks in random environment can 
be applied. When the graph is Z, Solomon's theorem shows that (directed and 
undirected) reinforced random walks are a.s. recurrent when ao(x, x+l) — ao(x, x— 
1) = ao > or bo(x,x + 1) = &o > 0. When the graph is the binary tree and 
bo(x,y) = bo > 0, the undirected reinforced random walk is transient for small A 
(or equivalently for large bo) and recurrent for large A (or equivalently for small 
bo). This last result was proved by R. Pemantle in [Pemlj . 

In this paper we address the question (posed by M. Bena'im to one of the authors) 
of what happens when the graph is Z and when on the event {X n = x}, 

F(X n+1 = x + l\F n ) = f ( , a ^ x + ^ ) , 

\a n (x, x - 1) + a n (x. x + I) J 

where / : [0, 1] — * (0, 1) is a smooth function. For general functions /, these 
walks are no longer random walks in random environment. So we have to use 
different techniques. But one can still attach to each site independent urn processes 
(generalized Polya urns). Under the assumption that the number of fixed points 
of / is finite, if the walk is recurrent, stochastic algorithms techniques show that 
for all x, a n {x, x + l)/{a n {x, x — 1) + a n {x, x + 1)) converges a.s. toward a random 
variable a x . This random variable takes its values in the set of fixed point of /. If 
ao(x, x + l) = ao{x,x— 1) = ao > 0, the sequence (a x ,x 6 Z) is i.i.d. Let us remark 
that Solomon's theorem states that the random walk in the random environment 
(a x , x £ Z) is a.s. recurrent if and only if E[ln(a 3 ;/(1 — a x ))] = 0. 

We focus here on cases when either / has a unique fixed point or all the fixed 
points are greater or equal to 1/2 and / > 1/2 on [1/2, 1]. We particularly study 
the case ao{x,x + 1) = ao{x,x — 1) = ao > 0. We give criteria for recurrence and 
transience: 

• when there exists one fixed point greater than 1/2 the walk is transient and 

• when 1/2 is the unique fixed point, depending on the initial condition ao 
and on the shape of / around 1/2, the walk can be either recurrent or 
transient. 

This last result shows that Solomon's criterion applied to the limiting values (a x , x G 
Z) does not determine recurrence versus transience. The proofs of the theorems 
given here involve martingale techniques inspired by the work of Zerner on multi- 
excited random walks on integers |Zerl IZer2] . 

The paper is organized as follows: in section 2 reinforced random walks are 
defined and their representation with urn processes is given. In section 3 are given 
the results on urns that are needed to prove the theorems given in sections 5 and 
6. A zero-one law is proved in section 4: recurrence occurs with probability or 1. 
In section 5 and 6 the case / > 1/2 and the case when there is a unique fixed point 
are studied. The last section develops some examples. 

2. Notation 

2.1. The urn model. We consider an urn model where balls of the urn are only 
of two types or colors, let say Red and Blue. Given a function / : [0, 1] — > (0, 1) we 
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alter the draw by choosing at each step a Red ball with probability f{a), if a is 
the proportion of Red balls. Then we put back two Red (respectively Blue) balls in 
the urn if a Red (respectively Blue) ball was drawn. In other words an urn process 
associated to / is a Markov process ((a n , l n ),n> 0) on [0, 1] x (0, +oo), where the 
transition probabilities are defined as follows: for all n, l n +i = Z n + 1 and a n +i is 
equal to (l n a n + 1)/(Z„ + 1) with probability f(a n ), or equal to l n a n / (l n + 1) with 
probability 1 — f(a n ). In fact a n represents the proportion of Red balls and l n 
the total number of balls in the urn at time n (at least if Iq and agio are integers). 
By abuse of notation we will sometime call the first coordinate (a n ,n > 0) an urn 
process associated to / (if there is no ambiguity on Iq). This model was introduced 
in [HLSj . and then further studied in particular by Pemantle |Pem4j . Duflo [D] 
and Benaim and Hirsch (see |BH] and [B]). We refer also the reader to the survey 
|Pem5j section 2.4, 2.5 and 3.2 for more details and references. 

2.2. Generalized reinforced random walks. Here we consider a particular 
model of (directed) reinforced random walk (X n ,n> 0) on Z where the evolu- 
tion is driven by urns of the preceding type on each integer. For other models 
see |Pem5] . A first way to define it is as follows. Let / : [0,1] — > (0,1) and 
(ag, Zjf)xez G ([0, 1] x (0, +oo)) z be given. Then if x G Z, set L% = and for n > 1 
let 

n-l 

L n '■= l {^ k =x}, 

be the total time spent in x up to time n — 1 by the random walk. Let then 



1 



n-l 

a l + l{X k =x,X k+1 =x+l} 



n ' IX I T x 1 1 / j *t^fc=X-,^Vfc + 

+ hn { fc=0 ) 

be the proportion of times it has moved to the right (up to some initial weights). 
Now if X n = x, for some x G Z and n > 0, then X n+ \ — x + 1 with probability 
f(otn) and X n+ i = x — 1 with probability 1 — /(afj. This defines recursively the 
random walk. Moreover for n > 1, define as the time of the n th return to x: 
for n > 1, = infjfc > t%_i : X^ = x} with the convention inf = oo, and 
Tq = infjfc > : Xu — x}. Set also 1* := Ifi + n, for n > 1. Let 

when t*_ x < oo, and = otherwise. Then the processes ((a£, Z^), < n < L^) 
form a family of urn processes of the type described above stopped at the random 
time L^. More precisely, {L^ > n} = {t* < oo} G T r x and on this event, 



a 



H< + 1 



n+l 



-l = x + ]\T T * 
= /(<£. ) = /«)• 



In the case the walk is recurrent, these urn processes are independent, and they are 
identically distributed when (ckq,Zq) does not depend on x. 

There is another way to define this random walk which goes in the other direction. 
Assume first that we are given a family of independent urn processes ((a£, Z£), n > 
0) xS z indexed by Z. One can consider this as the full environment for instance. 
Then given the full environment the random walk evolves deterministically: first 
it starts from Xq. Next let n > be given and assume that the random walk has 
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been denned up to time n. Suppose that X n = x and = k, for some k > 0. 
Then X n+ \ = x + 1 if a% +1 > and X n+ i = x — 1 otherwise. 

For n > 0, we define the environment w n € ([0, 1] x (0, +oo)) z at step n by 

w n := (af, s ,Zg + L^) x£Z . 

We denote by T n the a-algebra generated by (Xq,wo, . . . ,w n ), or equivalently by 
(wo, Xq, . . . , X n ). For i£Z and w some environment, we denote by E XiU) the law 
of the random walk starting from X — x and with initial environment w = w. If 
no ambiguity on x or w is possible we will sometime forget them in the notation. 
A random walk of law E x>t0 will be called a generalized reinforced random walk 
started at (x,w) associated to /. 

Observe that ((w n , X n ), n > 0) is a Markov process (whereas (X n ,n > 0) is not), 
and in particular: 

E x , w [g{X n+1 ) | T n \ = Ex„,»„b(^i)], 

for any (a;, w) and any bounded measurable function g. 

Note that the directed reinforced random walk started at xq, with initial weight 
(a (x, i/);ieZ, y £ {x — 1, x + 1}) and reinforcement parameter A defined in the 
introduction has law E XOiWo with f(x) = x and wg = (ag,/g) defined by 

x a (x,x + l) 

a n - 



a (x, x — 1) + ao(a;, ^ + 1) ' 
and 

_ ao(x, x - 1) + ao(x, x + 1) 
'o- ^ • 

The undirected reinforced random walk defined in the introduction has also the 
law of a certain generalized reinforced random walk. For example, the undirected 
reinforced random walk started at with initial weights bo{x, x + 1) = b > and 
reinforcement parameter A has law E , w g, with = (ag, Iq) defined by 

^=U'aJ and W °= L J 

This corresponds to the case when Ifi = l £ (0, +oo) for all x 7^ 0, af> = a € (0, 1) 
for x > 1, and = 1 — a , for x < —1. 

In the following w will satisfy: 

Hypothesis 2.1. TTie starting environment is such that for all x > 1, Wq — Wq. 

or will satisfy: 

Hypothesis 2.2. The starting environment is such that for all x > 1, = 
and for all x < — 1, = 1 . 

2.3. Hypothesis on / and stable points. Throughout the paper / : [0, 1] — > 
(0, 1) will be a regular function (C 3 is enough for our purpose). We say that p is 
a fixed point if f(p) = p. It is called stable if f'(p) < 1. We will assume that all 
fixed points of / are isolated. 
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2.4. Statement of the main results. Let X be a reinforced random walk of law 
Po,tu ; f° r some initial environment wo. This walk is called recurrent if it visits 
every site infinitely often, and it is called transient if it converges to +00 or to 
—00. We denote by R the event of recurrence, and by T the event of transience, 
T = {X n — » +00} U {X n — > —00}. In section [H it will be shown that, under 
Hypothesis 12.21 (or under Hypothesis 12.11 if / > 1/2), X is either a.s. recurrent or 
a.s. transient. 

The drift accumulated at time n by X is equal to 

14-1 

x k=0 

The methods developed in this paper are well adapted to the particular case / > 
1/2, making this drift nonnegative and nondecreasing. In this case one can define 
for all ieZ, 

00 

fc=0 

which is the drift accumulated at site x if the random walk visits x infinitely often. 
Then we have 

Theorem 2.1. Assume Hypothesis \2.1\ and that f > 1/2. Then the random walk 
(X n ,n > 0) is recurrent if, and only if, Ef^] < 1. 

Note that this theorem is an analogue of Zerner's criterion [Zerj for cookie random 
walks. Using the results of section I3~2l where the finiteness of Efdjj is discussed, 
this theorem implies in particular 

Corollary 2.1. Assume Hypothesis \2.1\ and that f > 1/2. If 1/2 is not the unique 
stable fixed point of f , or if f" (1/2) > 0, then (X n , n > 0) is a.s. transient. 

Proof. Let p be a stable fixed point of /, with p ^ 1/2. As states Theorem 13.11 
below, a\ converges to p with positive probability. Thus, with positive probability 
5^ = 00. When p = 1/2 is the only fixed point and if /"(1/2) > 0, Proposition ^. 41 
below shows that 8^ = +00 a.s. and we conclude by using Theorem 12. 11 □ 

Without the assumption that / > 1/2 , we will prove the 

Theorem 2.2. Assume Huvothesis \2.2\ and that f has a unique fixed point p. 

• If p^ 1/2 thenP[R] = 0. 

• Ifp = 1/2 and f (1/2) = 0, thenE^} > 1 orE^ 1 ] < -1 imply W[R] = 0. 

Notice, what is part of the result, that 8%^ and E[(5^] are still well defined for all 
x under the hypothesis of the theorem. 

The sufficient condition to get ¥[R] = in the case p = 1/2 has to be compared 
to the result of [KZerj in the context of cookie random walks, where it is proved 
that this is also a necessary condition. Here we were not able to prove this. 

Theorem 12.21 implies in particular 

Corollary 2.2. Assume Hypothesis \2.2\ and that 1/2 is the only fixed point of f . 
If f (1/2) = and f"(l/2) ^ 0, then ¥[R] = 0. 

Proof. This follows from Proposition 13.41 which shows that 5^ — +00 a.s. if 
/"(1/2) > 0, and tf- 1 = -00 a.s. if /"(1/2) < 0. □ 
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These results allow to describe interesting phase transitions. This will be done 
in the last section. For example, there exists a function / > 1/2 having 1/2 as a 
unique stable fixed point, such that if X has law P = Po.to OJ with w'q = (1/2, 1) for 
all x, then 

Theorem 2.3. There exists l\ > such that ifl>h the walk is recurrent whereas 
if I < l± it is transient. 

This phase transition is similar, yet opposite, to the one observed by Pemantle 
for edge-reinforced random walks on trees [Pemlj : there exists Ai > such that 
if the reinforcement parameter A is smaller than Ai it is transient whereas it is 
recurrent when this parameter is greater than Ai. Indeed, in the non-oriented rein- 
forced framework discussed in the introduction, starting with small I is equivalent 
to starting with large A. 

3. PRELIMINARIES ON URNS 

3.1. Convergence of urn processes. We recall here some known results about 
convergence of urn processes. In particular the next theorem is of fundamental 
importance in all this paper. Remember that all functions / considered in this 
paper satisfy the hypothesis of section [2~3l 

Theorem 3.1 ( [HE5] , [Pem4| 1. Let (a n ,n > 0) be an urn process associated to 
some function f . Then almost surely a n converges to a stable fixed point of f and 
for any stable fixed point, the probability that a n converges to this point is positive. 

The convergence to a stable fixed point p with positive probability was first 
proved in |HLS| , when f(x) — x changes of sign near p, and in |Pcm4j in the special 
case when the sign of f(x) — x is constant near p. The non existence a.s. of other 
limiting points was also first proved in |HLS| (for extensions to more general settings 
see [D], [B], |Pem2j . [Pem3] h 

There is also a central limit theorem, which can be extracted from the book of 
Duflo: 

Theorem 3.2 ([D] Theorem 4. III. 5). Suppose that p E (0, 1) is a stable fixed point 
of f . Let a = f'(j>) and v 2 = p(l — p). If a < 1/2, then conditionally on a n — > p, 
\/n(a n — p) converges in law, as n tends to +oo, toward a normal variable with 
variance v 2 /(I — 2a). 

3.2. Convergence of the drift. For n 6 N, we set 

n 

5 n = ]T(2/K) - 1). 

k=Q 

Then S n will correspond to the drift accumulated at a given site after n + 1 visits 
to this site. If 8 n converges when n — > +oo, we denote by 5^ its limit. We will 
need also to consider its negative and positive parts defined respectively by 

n 

5- :=^(2/K)-l)_, 

k=0 

and 

n 

5+ :=]T(2/K)-1) + , 

k=0 
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for all n > 0. In fact we can always define in the same way 5^ and even when 
S n does not converge. Moreover observe that if E[<5^,] or E[<5+J is finite, then S„ 
converges a.s. and E[<5 co ] = E[5+] — E[£~]. It happens that for our purpose, such 
finiteness result will be needed. 

The problem is that the convergence of the drift appears to be a non-trivial 
question. To be more precise, we were able to obtain a satisfying result essentially 
only when / has a unique fixed point. When this fixed point is greater (resp. 
smaller) than 1/2, it is immediate to see that the drift converges a.s. toward +oo 
(resp. — oo). However to see that E[(5^] (resp. E[<5+J) is finite, some non-trivial 
argument is needed. Since it is the same as in the more difficult case when 1/2 
is the unique fixed point, we start by this case. Let us give here an heuristic of 
how we handle this convergence problem when p = 1/2: the central limit theorem 
(Theorem 13. 2j) shows that \k(a.k — 1/2) converges in law. A Taylor expansion of 
/ shows that 

(1) 2/K) - 1 = 2/'(l/2)K - 1/2) + /"(1/2)K - V2) 2 + 0((a k - 1/2) 3 ). 

The first term is of order fc -1 / 2 , the second of order k~ x and the third of order 
fc-3/2_ whcn /'(1/2) ^ 0, then E[S~] = E[<5+] = oo (see Proposition O below). 
When f (1/2) = and /" (1/2) > 0, 5+ = oo and since (2f(a k ) - 1)- is of order 
fc~ 3 / 2 , E[<5-] < oo. Finally, when /'(1/2) = /"(1/2) = 0, both E[<5"] and E[<5+] 
are finite (see Proposition 13. 21 below). 

Proposition 3.1. Assume that 1/2 is the unique fixed point of f and that /'(1/2) ^ 
0. Then E[S X ] = E[S+] = +oo. 

Proof. Let us prove that E[5+ ] = +oo (the proof of the other equality E[5<^,] = +oo 
is identical). Since /'(1/2) ^ 0, there exist positive constants cj., C2, such that 

(2) E[(2/(a fc )-l)+] > Cl E[K-l/2)+l {(Qfc _ 1/2)<C2} ] 

(3) > [b/kictk ~ l/2))+l { ^ (afc _ 1/2)<1} 

for k large enough. Then the central limit theorem (Theorem l3.2[) gives ^ fc E[(2/(ajfe)- 
1) + ] = +oo. This proves that E[<5+J = +oo, as wanted. □ 

Proposition 3.2. Assume that 1/2 is the unique fixed point of f and that /'(1/2) = 
0. Then 

• If f" '(1/2) > 0, then E[<5"] < +oo. 

• Iff" (1/2) < 0, then E[<5+] < +oo. 

• If f" (1/2) = 0, thenE[S^} and E[<5+ ] are both finite. 

In all cases, S n converges a.s. toward 5^ and E[(5oo] is well defined. 

Proof. For x € [-1/2, 1/2], let h(x) := f(x + 1/2) - x - 1/2, and for n > 0, let 
x n := a n — l/2. Let also e n +i be equal to 1 if a n +i > ct n and equal to —1 otherwise. 
By definition (x n ,n > 0) satisfies the following stochastic algorithm: 

(4) x n+1 =x > k{Xn) > 



lo + n + 1 la + n + 1 
where £ n+1 = e n+1 - E[e n+i | T n ]. 
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• Consider first the case /"(1/2) ^ 0. Then the sign of / — 1/2 is constant in a 
neighborhood of 1/2. To fix ideas let say that f > 1/2 in [1/2 — e, 1/2 + e] for some 
constant e > 0. In this case we will prove that < +oo. For all n > 0, 

n 

E[5-]<^P[4>6 2 ]. 
k=a 

Therefore it suffices to prove that this last series is convergent. This will be achieved 
by using Equation (j4|) and some ideas from the proof of Proposition 4.1.3 in [D] , 
First, since 1/2 is the unique fixed point of /, there exists some constant a > 
such that xh(x) < —ax 2 /2 for all x £ [—1/2,1/2]. Moreover we can always take 
a < 1/2. Next Equation g]) gives 

x n+1 <x n (l-— — ) + 



l Q +n + r lo + n+1 (h+n + l) 2 ' 

where w n+1 is bounded, i.e. there exists C > such that a.s. |« n +i| < C for all n. 
Let s„ = nl-=o(l — + k + 1)). By induction we get 

n n 

x 2 n < s n xl + s n 53((I + k + l)s k )- 1 x k ^ k+1 + Cs n ^2((l + k + lfsk)- 1 . 

k=0 k=0 

Since s k ~ k~ a , this gives 



x 2 n < C's n + s n y^X(k + k + l)s k ) 1 x k £k+i, 

k=0 

for some constant C > 0. Define the martingale (M n ,n > 0) by 

n 

Mn = £((io + * + iHr^Cfc+i Vn > 0. 

fc=0 

Then for n large enough and any integer a > 0, we have 

(9i \ 2a 

P[xl > e 2 ] < P[ S „|M„| > e 2 /2] < i-^-E[M 2 "]. 

But Burkholder-Davis-Gundy inequality (see e.g. [W] p. 151) implies that for some 
constant c a , 

E[M 2Q ] <c q E[< M >£], 
where < M >„:= ^fe =0 ((io + fc + l)sfe)~ 2 a;^ 2 +1 . Moreover, since a < 1/2, 

n 

< M >„< 5^((i + k + l)s fe )- 2 < C, 

fe=0 

for some constant C > 0. Thus 

s 2Q £[< M >£] < C Q n- 2aa , 
with C a > a constant. Taking now a large enough shows that 

+00 

£>[a£ >e 2 } <+oo, 

as we wanted. 
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• It remains to consider the case when /"(1/2) = 0. There exists a constant 



C > such that 
from which we get 



|2/K)-1| <C\x n \ A Vn>0 



S-+5+<C^2\x k \ 3 . 

fc=0 

Thus it suffices to prove that J2k=o ^[l^fel 3 ] i s finite. But since /'(1/2) = 0, there 
exists e > such that 2xh(x) < —x 2 when \x\ < e. Therefore (01 gives in fact 

2 , 2w ^ P\\x n \ > el C 

Now the proof of the preceding case shows that 

E[x 2 n+1 ]<E[x 2 n ](l--) + ^. 

This proves by induction that E[x 2 /\ < C/n for some constant C > 0. Let us now 
consider the moments of order 4. Since 4x 3 h(x) < ~3x 4 in [— e, e] for some e > 0, 
(|4]) gives similarly 

E[4 +1 ]<E[4](l-i) + §, 
n n z 

for some constant C > 0. By induction, this gives i?[x^] < C'nT 2 , with C" > 
another constant. Then Cauchy-Schwarz inequality gives (up to constants) 

E[K| 3 ]<(E[^]E[4]) 1 /2< n -3/ 2j 
which is summable. This finishes the proof of the proposition. □ 

Observe that the argument given in the proof of the above proposition applies 
as well when the unique fixed point of / is different from 1/2. Thus we proved also 
the 

Proposition 3.3. If f has a unique fixed point p > 1/2, resp. p < 1/2, then E[£^], 
resp. E[<5jJ, is finite. In particular S n converges a.s. toward +oo, resp. — oo. 

Our last result concerns the a.s. non-finiteness of 5^. First if p ^ 1/2 is a stable 
fixed point of /, then conditionally on {a n — * p}, 5 n /n converges toward 2p — 1 
and thus Soo = +oo. The next result investigates the case p = 1/2. 

Proposition 3.4. 7/1/2 is a stable fixed point of f, /'(1/2) = and f"{l/2) > 
(respectively /"(1/2) < 0), then conditionally on a„ — > 1/2, almost surely = 
+oo (respectively Sco — —oo). 

Proof. To fix ideas assume that /"(1/2) > 0. The other case is analogous. A 
limited development of / near 1/2 gives 



(5) 



5 n -/"(l/2)» fe -l/2) 



k=0 



< C*^K-1/2| 3 Vn>0, 



k=0 



with C > some positive constant. For n > 0, we set z n := y/n(a n — 1/2). We 
already saw in Theorem 13.21 that conditionally on {a n — > 1/2}, z n converges in law 
toward a normal variable. In fact this holds in the sense of the trajectory. More 
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precisely, an elementary calculus shows that (z n ,n > 0) is solution of a stochastic 
algorithm of the form: 

6 z n+1 =z n -—— — + —===, 

la + 7i+ I ^/l + n+l 

where r n+ i = 0(^/n(a n - 1/2) 2 + n^ 1 ). For t £ [logn, log(n + 1)], let 

(7) Y t = z n+1 + (t-logn)(-z n+1 /2 + r n+1 ) + (t - log n) 1/2 £ n+2 . 

For u > 0, call (Y^ , t > 0) the continuous time process defined by = Y u +t 
for t > 0. Then Theorem 4. II. 4 in [D] says that (conditionally on {a n — * 1/2}) the 
sequence of processes (Y^ , t > 0) converges in law in the path space toward an 
Ornstein-Uhlcnbeck process (U s , s > 0), when u — * +oo (the condition on r n in the 
hypothesis of the theorem is not needed here, as one can see with Theorem 4. III. 5 
and its proof in [D]). Now we will deduce from this result that a.s. on the event 
{a„^l/2}, 



E 

fc=i 

If we define z t for all t > by z t = zm, then one can check that Y]^L zf./k 
is comparable with J^~°° zf/t dt — J^ 2 z 2 t dt. So if this series is finite, then 
J n+ z e' dt ~* when n — > +oo. Moreover, using © and ([7]), we have that 
a.s. on the event {a n — > 1/2}, Y" t 2 = z 2 t + o(l). Therefore a.s. on the event 
{Efc z fe/ fc < oo} n {a n -> 1/2}, we have £ +1 Y t 2 dt -> when n — > +oo. But this 
cannot hold since on the event {a n -> 1/2}, £' + F t 2 converges in law toward 
C/ 2 ds, which is a.s. non-zero. Thus holds. Finally, using the fact that a.s. 

on the event {a n -> 1/2}, ^Lo W ~ V 2 ! 3 = ° (ELi 4) > © and ® show that 
(5oo = +oo. □ 

4. A ZERO-ONE LAW 

In all this section X is a generalized reinforced random walk of law P = Po lU > 
associated to /, and wq satisfies Hypothesis l2.2l We will try to relate its asymptotic 
behavior with urn characteristics. Our first result is general. It is a zero-one law 
for the property of recurrence. Remember that the random walk is recurrent if all 
sites are visited infinitely often. 

Lemma 4.1. Assume Hypothesis\EM Then F[R] € {0, 1},. and P[T] = 1 - F[R). 

Proof. First Borel-Cantclli Lemma implies that if a site is visited infinitely often, 
then the same holds for all sites. So there are only three alternatives. Either the 
random walk is recurrent, or it tends toward +oo or toward — oo. In other words 
F[T] = 1-F[R}. Thus, if 

T n = inf{fc >0\X k =n} Vn > 0, 

then l{ Tii<+co j converges toward 1_ru{x„^+oo}j when n — > +oo. In the same way 
the event {X n > Vrt > 0} is included in {X n — > +00} . In fact there is a stronger 
relation: 

Lemma 4.2. For any initial environment wq and any k > 0, PLY n — > +00] > if, 
and only if, Pfe )U)0 [X„ > k; Vn > 0] > 0. 
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Proof. Wc do the proof for k = 0. The other cases are identical. This proof is 
similar to Zerner's proof of Lemma 8 in [Zerj . We just have to prove the only if 
part. Call t 2 the last time the random walk visits the integer 2. If C is some path 
of length k starting from and ending in 2 on Z, call Ec the event that the random 
walk follows the path C during the first k steps. Define also wc as the state of all 
urns once the walker has performed the path C. If P[X n — > +oo] > 0, then for some 
path C from to 2, we have 

< P[E c ,X n > 2 Vn > k] = ¥[E C ] x F 2 ,w c [X n > 2 Vn > 0]. 

Now construct C as follows: it starts by a jump from to 1 and then we add (in 
chronological order) all the excursions of C above level 1. Then clearly 

Pa,« c [X n > 2 Vn > 0] = ¥ 2 , Wc , [X n > 2 Vn > 0]. 

Moreover, since the range of / is in (0, 1), it is elementary to see that P[i?c'] > 0. 
Thus 

P[X n > Vn > 0] > P[Ec>] x P 2 .w c , [X n > 2 Vn > 0] > 0, 
which proves the lemma. □ 



We can finish now the proof of Lemma 14. H The martingale convergence theorem 
and the Markov property imply 

Ux n ->+oo} = I™ Po,w [X n -> +0O | ^T„]l{T„<+oo} 
n— >+oo 

= lim P n w T \X n — ► +oo]l/ T l<+oo> 

n— *+oo 

> limsupW n>WTn [X m > n Vra > 0]l {Tn<+oo} 



1,11)0 



[X n >l Vn > 0]l flu{Xii ^ +oo} . 



Then multiply the left and right part of this inequality by 1r and take expectation. 
This gives 

(9) P hW0 [X n >l Vn > 0] P[R] = 0. 

In the same way we have 

P-i,™ 1** < - 1 Vn > o] p^] = o. 

These two equalities and Lemma 14.21 prove the lemma. □ 



Remark 4.1. Let Tq be the first return time to 0. Then the usual equivalence 
Tq < +oo a.s. if and only if is a.s. visited infinitely often, is true. Indeed if 
Tq < +oo a.s. then by Lemma \^.'i\ a.s. X n does not converge toward ±oo. Then 
the 0—1 law says that R holds. 

5. The case with only non-negative drift 

Here we assume that / > 1/2 and that wq satisfies Hypothesis 12.11 In the 
following X is a reinforced walk of law P = Po,«, . In this case we have a more 
precise zero-one law. 

Lemma 5.1. Assume Huvothesis \2.1\ and f > 1/2. We have the alternative: either 
(X n ,n > 0) is almost surely transient toward +oo, or it is almost surely recurrent. 
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Proof. Since / > 1/2, at each step the random walk has probability at least 1/2 to 
jump to the right. Thus an elementary coupling argument (with the usual simple 
random walk on Z) shows that a.s. the random walk does not converge toward 
— oo. We conclude with © (which holds when assuming only Hypothesis 12. ip and 
Lemma 14.21 □ 



Remark 5.1. We notice here that the hypothesis / < 1 made in section 1231 is 
not needed when / > 1/2. Indeed the only place where it is used is in the proof 
of Lemma [4.21 to show that P[£e'] > 0, but the reader can check that this is not 
needed when / > 1/2. This remark will be of interest for the last section. 

Proof of Theorem \2.1[ We follow essentially the proof of Theorem 12 in [Zer] . Let 
us recall the main lines. First we introduce some notation. For n > 0, let 

U n= ^2 Mx k =o, x k+1 =-i}, 

k<n-l 

and let 

X n = X] ( X k+1 - X k)l{X k >0}- 
fc<n-l 

A straightforward computation gives the equation 

(10) X+ = max(X„, 0) - U n Vn. 
We define the drift accumulated in x up to time n by 

D*=£>/K)-l), 

k=0 

and the drift Z?+ accumulated in the non-negative integers by 

x>0 

Let (M+,n > 0) be the process defined by 

M + = X + - D + Vn. 

n n n v 

It is a basic fact that (M+,n > 0) is a martingale. In particular for all a > and 
all n > 0, using ([i"0|) with the martingale property, 

E[max(X TaA „, 0)] = E[t/ TaA „] + E[£>+ A J, 

where 

T a = inf{fc >0\X k = a}. 

Now Lemma [5Tl implies that T a is a.s. finite. Moreover (U n , n > 0) and (D+, n > 0) 
are non-decreasing processes. Thus letting n go to +00 gives with the monotone 
convergence theorem 

(11) a = E[U Ta ] + ^[DtJ Va>0. 

Moreover the Markov property shows that for any integer x € [1, a], 

E[BfJ = m x , WTx [D x Ta ]] = Ei, Wo [D^_ x+i ], 

where the last equality holds because for all y > x, w\ — w\. Moreover E[Dj, ] 
and Ei iWo [Drp _J differ at most by E[iV], where N is the number of visits to 
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before the first visit to 1. Since the probability to jump from to f is bounded 
away from 0, E[iV] is finite. Therefore 

lim -E[D+] = lim - I E 0<Wo [D° T ] + Y> 1)U)0 [D l Tx ] ] 

= Ei,«o[^]- 
Then (fTTj) gives the inequality 

E[£>^] < 1. 

So if the random walk is recurrent, almost surely — 8^, and E[<J;U < 1. This 
gives already the only if part of the theorem. 

Assume now that the random walk is transient. Then Lemma l4. 21 shows that 

nC - Dl] > cE[8l - Si], 

where c = ¥i <W0 [X n > 1 Vn > 0] > 0. Now since the sequence (<5^)„> is non- 
decreasing, if E[(J^ — Sq] was equal to 0, this would mean that a.s. <5* = Sq for 
all n. In other words the walk would evolve like the simple random walk, which 
is recurrent. This is absurd. Thus EfdjJ > E[_D^]. It remains to prove that 
£[1?^,] = 1. From (fTT]) we see that it is equivalent to prove the 

Lemma 5.2. If the random walk is a.s. transient, then lim a ^ +00 E[£/T a ]/a = 0. 

Proof. This lemma can be proved by following the argument of Lemma 6 in |Zerj , 
that we reproduce here. For i > 1, let 

o-i = inf{j > Ti | X s = 0}. 

We have E[U Ta ] = Y%Zo M U n +1 - U Ti ]. Next U Tl+1 - Z7 T . ^ only on the set 
Aj := {cr,j < T i+1 }. Moreover (jTTJ) holds for any starting environment. Thus 

E[U Ti+1 - U Tt ] < E^Eo,^ [U Tl+l \] <{i + 1)P[A], 
for all i. It remains to prove that 

(12) iVzPL-LJ^O, 

when a — > +oo. Let Yi = ¥[Ai | ^rj- Since the random walk is transient, the 
conditional Borel-Cantelli lemma implied 

(13) ^2 Yi < +°° a - s - 

i>l 

Moreover a coupling argument with the simple random walk and standard results 
for this random walk show that a.s., Yi <\/i for all i. Let e > 0. For all i, 

¥[Ai] = E\Yil {Yi<e/i} ] +E[Y i l {Yi > e/i} ] < ~ + F[Yt = e/l] . 



since we were not able to find a reference we give here a short proof: let (H n , n > 0) be 
the (y r T„)n>0 martingale defined by H n := 5Z"=i ^-A- ~ Yi, for n > 0. Let I > 1 and let 
T ; ' = inf{fc | H k > I}. Then H nAT r a.s. converges toward some limiting value q ; £ R, when 
n — » +oo. If a.s. only a finite number of Ai's occur, then a.s. T ; ' is infinite for some I > 1. This 
implies the desired result. 
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So we can divide the sum in IT2|) in two parts. One is lower than e and the other 
one is equal to 

i r " 

Li=l 

But since (fl3| holds, a.s. the density of the i < a such that Yi > e/i tends to 
when a tends to +oo. Thus the preceding sum converges to 0. This concludes the 
proof of the lemma. □ 

This completes the proof of Theorem 12. II □ 

In section [7] we will sec different examples of functions / > 1/2, symmetric with 
respect to 1/2 which show in particular that in the case when 1/2 is the only stable 
fixed point and /"(1/2) = 0, both regimes (recurrence and transience) may appear. 

6. The case with a unique fixed point 

Here we do not assume anymore that / > 1/2, but we assume that / has a unique 
fixed point. The initial environment satisfies Hypothesis 12.21 and still P = Fq iWq . 



Proof of Theorem \2.SH The idea of the proof is the same as for Theorem [2TTJ How- 
ever a priori we have to be careful when taking limits since the drift (Z?+)„>o is 
not anymore a non-decreasing function. But for any integer x > 0, Proposition 
13.21 and Proposition 13.31 show that E[D^] converges toward E[_D^]. In fact since 
E[<5;x)] < +°° or E[5+] < +oo, if we replace n by any increasing sequence of stop- 
ping times r n converging toward t^, these propositions show that E[D^ ] converges 
toward E[£>^]. So in fact we get 

lim E[D+ A J=E[£>+J. 

Next observe that for all a > and n > 0, Xi . < a. Thus, using that (M+) n >o 
is a martingale, we have 

E[D+ ] < a Va > 0. 
Assume P(R) — 1. Then the Markov property implies that if 1 < x < a, 
(14) E[ZJf-J = E x , Wo [D^ a ) = E l!Wo [D^J. 

Letting a tend to +oo in (TH| . and using the fact that D^, = YTx=o gi yes 

Ei, Wo [Dlo\ < 1- 

Since ¥[R] = 1, a.s. = 5}^ and we have E[<J^J < 1. The other inequality 

Efc 1 ] > —1 is similar. □ 

Let us state now the following standard monotonicity argument: 

Lemma 6.1. Let f < g be two functions. Then there exists a coupling of two urn 
processes ((a n ,l n ),n > 0) and {(f3 n ,l' n ),n > 0) associated respectively to f and g, 
such that lo = 1' , ctQ — (3q, and almost surely a n < (3 n for all n > 0. 

Proof. The proof is standard. Let (Ui,i > 0) be a sequence of i.i.d. random 
variables uniformly distributed on [0, 1]. We define two urn processes starting with 
initial conditions like in the lemma. Then at step n, a n +i > ct n if, and only if 
f{ct n ) > U n . The same for f3 n +i (with g in place of /). Assume now that for some 
n, a n > (3 n . Assume also that n is the lowest index where such inequality occurs. 
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This means that a n -i = fin-i- But since / < g, by definition of our processes, we 
get an absurdity. □ 

This lemma together with Theorem 12.21 allows to consider also the case when / 
has possibly more than one fixed point but under the condition / > 1/2 on [1/2, 1]. 
More precisely we have 

Corollary 6.1. Assume that Hvvothesis \ 2. 6 2\ holds, that f > 1/2 on [1/2,1] and 
that all fixed points of f are greater or equal to 1/2. 

• If 1/2 is not a fixed point, then W[R] = 0. 

• If 1/2 is a fixed point, but not the only fixed point, and /'(1/2) = 0, then 
V[R] = 0. 

Proof. If any of the two hypothesis of the corollary is satisfied, then there exists a 
function g such that g < /, g has a unique fixed point equal to 1/2, and g'(l/2) = 0. 
We can also assume that g is increasing on [0,1/2]. Applying Lemma [6TT1 we see 
that there exists an urn process (f3 n , n > 0) associated to g such that /3 n < a n for 
all n. Now the proof of Proposition 13.21 shows that 

53E[(2s(A0-l)_]<+oo. 

n>0 

Since g is increasing on [0, 1/2] and f > 1/2 on [1/2, 1], this implies that E[<5^] is 
finite. Moreover we know that = +oo a.s. So we have everything to apply the 
proof of Theorem 12.21 and to conclude. □ 

7. Some examples 

Our goal here is to give examples of functions / leading to interesting behavior 
for the associated random walk, in view of the previous results. In all this section 
we consider a function /, symmetric with respect to 1/2, i.e. such that f(l/2 — x) — 
/(1/2 + x) for all x € [0, 1/2], decreasing on [0, 1/2] and increasing on [1/2, 1]. Wc 
assume also that / has a unique fixed point, equal to 1/2, and that /"(1/2) = 0. 

We start now by a comparison result. Let u be some positive real number. Define 
/„ by the equation 2f u — 1 = [u(2f — 1)) A 1. One can see immediately that f u has 
the same properties as / for all u, and moreover that f u < f v if u < v. Denote by 
((a",ZJ£),n > 0) an urn process associated to /„ such that (oq,Iq) = (1/2,1) with 
I > 0, and set <5« := E„>o( 2 /«) ~ !)• Then we have the 

Lemma 7.1. For all u, E[J^] < +oo. The maps u h-> E /u and u h-> E[<5^], 
are nondecreasing respectively on (0, 1] and on [0, +oo). In particular E[<^] — > 0, 
when u — > 0. Moreover E[<J^J — > +oo, when u — > +oo. 

Proof. The first claim results from the proof of Proposition 13.21 For the second 
claim, consider first < u < v < 1. By symmetry, for any k > 0, 

E[2/„K) - 1] = 2E[(2/„K) - l)l {aJ >i /a} ]. 

Moreover, since /„ is nondecreasing on [1/2, 1] and since one may couple ctf, and 
a v k such that a\ < a v k a.s. by Lemma IBTTl 

E[(2/ u (aJJ) - l)l {aS >i/2}] = -E[(2/„(aJ|) - l)l {a «>i /a} ] 

< -E[(2/„(a2)-l)l {a .> 1/a} ]. 
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The result follows by summation. 

The fact that u i— » E[5^] is nondecreasing on [0, +oo[ is similar. It remains to 
find the limit when u — > +00. For this, fix some n > 1. Then one can observe that 
there exists e > 0, such that |a 2fc+1 — 1/2 1 > e for all k < n. This implies that for 
u large enough, E[<5^] > n/2. Since this holds for all n, the result follows. □ 

The preceding lemma and Theorem 12.11 show that there is a phase transition: 
let X be a generalized random walk started at (0,u>o) associated to f u , where the 
initial environment is such that Wq = (1/2, 1). Then there exists some uq > such 
that for u > uq, the random walk associated to /„ is transient, whereas for u < u 
it is recurrent. In particular recurrence and transience may both appear. The 
question of what happens at uq is related to the continuity of E^^] with respect 
to /. But explicit calculus show that if u — > uq, then for all n, E[5™] — > E[5™ ]. 
Together with the monotonicity of E[<$^>] in u, this proves that E[<5^] is continuous 
in u. In particular for u — u$ the random walk is recurrent. 

Our second problem concerns what happens when initial conditions vary. Here also 
we will see that there is possibly a phase transition. For (a, I) £ [0, l]x]0, +00 [, we 
denote by E aj ; the law of an urn process starting from (a, I). First let us prove the 

Lemma 7.2. Let a £ [0,1] \ {1/2} and I £ (0, +00), be such that 2al - I £ N. 
Then E a>2 ;[c>oo] > E 1/2 ,2i[^oo]- 

Proof. We use a standard coupling argument. Let (Ui)i>o be a family of i.i.d. 
random variables, uniformly distributed in [0,1]. Let start two urn processes 
(a n ,n > 0) and (/3 n ,n > 0), respectively from (a, 2/) and (1/2,2/). They evolve 
according to the following rule. If at step n, a n or f3 n is equal to x > 1/2, then we 
add one Red ball in the corresponding urn if U n < f(x). Now if x < 1/2, then we 
add a Red ball if U n > 1 — f(x). The condition 2al — I £ N assures by induction 
that l n a n — l n Pn S Z for all n > 1. This in turn shows that the two urn processes 
(as well as their symmetric with respect to 1/2) cannot cross each other without 
meeting them. Thus for all n > 0, \/3 n — 1/2| < \a n — 1/2 1 . The lemma follows. □ 

The preceding results show in particular that the property of recurrence or tran- 
sience may depend on the initial conditions of the urns (even if Iq is fixed) . Indeed 
it suffices to consider / such that Ei/2,2i [800] — L which is possible by Lemma FTTI 
and the continuity in u of E[^] as explained above. Then the preceding lemma 
shows that for any a / 1/2 satisfying the condition of the lemma, the random 
walk associated with urns starting from (a, 21q) is always transient, whereas it is 
recurrent if they start from (1/2, 2io)- 

We arrive now to our last result. 

Lemma 7.3. The map I 1— * E]y 2 ,2([^oo] * s continuous on (0, +00), non-increasing, 
and converges toward when I — ► +00. 

Proof. The continuity of the map is similar to the continuity of E[5£J in u, observed 
above. The fact that the map is nonincreasing can be seen by using a coupling 
argument like in the preceding lemma. Indeed let Iq < l-y. Let (a n ,n > 0) and 
[Pni n > 0) be two urn processes starting respectively from (1/2, 2l\) and (1/2, 2/ ). 
Define their joint law like in the previous lemma. Observe that each urn process 
cannot jump above 1/2 without touching it. In the same way, if for some n, 1/2 < 
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Pn+i < (i n and 1/2 < a n +i < a n < n , then 

Pn 



J n+1 — i i i i ' 

2« + n + 1 
and 



2*i + n 

Thus 

Pn — Pn+1 < Pn — «n.+l- 

In other words the two urn processes cannot cross each other without meeting them. 
Thus for all n > 0, \/3 n — 1/2 1 > \a n — 1/2|, which proves the desired result. It 
remains to find the limit when I — ► +oo. But for each n, Ei/2.2z[<^n] converges to 
when I — > +oo. Since moreover for fixed I it converges toward Ei/2,2i[^oo]) when 
n — > +oo, the result follows. This finishes the proof of the lemma. □ 

We finish by the 

Proof of Theorem \2.3[ It suffices to choose / and l such that E 1 / 2 ,2; [^oo] > 1- 
Then the result follows immediately from the preceding lemma. □ 
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